A predictive model may also be known as a “statistical model”, a “machine learning model”, an “artificial intelligence model”, or by some other name. The important feature of these models is their use of one or more variables, known as inputs or independent variables, to calculate a value for another variable, known as a target or dependent variable. The method of calculation of the target variable is dependent on a type of predictive model. Many modern predictive models provide no means of interpreting their prediction results. The relationship between each input variable and the resulting target variable value is not readily discernible by the user of the predictive model. This inability to explain or interpret the predictive model results causes the user of the model to have lower confidence in how well the model is performing its task. Explaining or providing an interpretation of the predictive results may also provide insights into the operation of the system being modeled by the predictive model.